The observed color of an image is formed from the spectral energy distributions of the light reflected by the surface reflectance, and the intensity of the image color is determined by the imaging geometry. Diffuse reflection can be assumed to be associated only with the relative angle between the light direction and the surface normal in the imaging geometry regardless of the viewing direction. Specular reflection, on the other hand, is dependent on the viewing direction. This viewing dependency of the specular reflection can lead to problems in many computer vision applications such as stereo matching, segmentation, and/or recognition.
Stereo matching extracts of 3D information from digital images by comparing information about a scene from two vantage points. Because specular reflection changes depending on the view point, if one of the two images under comparison to extract 3D information includes specular reflections and the other does not, large intensity differences may be present. This can lead to miscalculations of 3D information from these two images. Despite this potential for error, it is common to assume that specular reflections do not exist in the images.
Image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain visual characteristics. A specular reflection within an image shows the reflection rather than the object upon which the reflection occurs. As such the color and/or intensity of pixels representing the specular reflection may not be labeled with the object, which is the object of image segmentation. This can lead to pixels being mislabeled or segmented. Similar problems can occur in image recognition techniques.
Most applications simply consider the observed image as a diffuse reflection model and disregard the specular reflection components because they are considered outliers.